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ABSTRACT 


In this paper constant feedback control laws are investi- 
gated for linear sampled-data systems. Derivations for the 
minimum-time problem and the infinite stage regular problem 
which yield constant feedback optimal control laws are 
presented. A case study for the finite stage regulator 
problem indicates that constant feedback control, although 
suboptimal, may be feasible from a design viewpoint. Several 


numerical examples illustrate the results. 
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1. INTRODUCTION 
1.1 General 

Optimum performance implies that a system performs in 
the "best" possible manner according to some specified set 
of criteria. If a criterion is established in a mathematical 
form, called a performance index, the "best" performance results 
when this index 1S minimized or maximized. An optimum control- 
ler then generates the sequence of control signals necessary 
to obtain an extremum of the performance index. 

This thesis investigates the optimal synthesis of linear 
sampled-data control systems in order to determine an optimum 
or “suboptimum" control law in the form of stationary, linear 
feedback of the plant variables. 

If the performance index is specified a a quadratic 
£unetionalwon the plant variables and the control variables, 
for a linear system, the optimum control law that is derived 
to obtain an extremum of this functional generally results in 
aainecw ems var yingefunmct ionsofethesplant variables, Gitte, 
multiloop feedback. The physical realization of a time-vary- 
ing control law requires a digital computer, or some other 
time-varying device. If a stationary control law can be 
obtained, the complexity of design, weight and cost of the 
controller is reduced. This is the motivation for this 


investigation. 


Tt will be shown in two cases that the optimum control law 
is stationary. In a third case the optimum control law is 
time-varying, but a case study shows that a stationary control 
law can be obtained that is "close" to optimum in performance. 
IZ SYStem DeScripeon 

The physical system to be controlled is called the plant, 
and is described by a set of linear differential equations 
with constant coefficients. It is controlled by certain 
physical quantities, called inputs or controls. The physical 
variables of the plant which can be measured directly are 
called outputs. 

By definition, the state of a dynamic system is the 
smallest collection of numbers which must be specified at time 


C= along with the control for time t2t,, in order to be 


Oo! 
able to predict the behavior of the system for any time t St,. 
In explanation, suppose the plant, which is described by an 
nth order differential equation, is simulated by an analog 
computer. This requires exactly n integrators. The outputs 


KypreeeeX of the n integrators, measured at any instant of 


Mi 


timé, are the staté of the plant at that inStant. The State 


may be regarded as a vector (nxl matrix) and is written as 


ed 


mat) = x, (t) 
(4) | 
where the components Xz (t),..-,x, (t) are the state variables. 


LO 


The plant can be described by the state variable form of 

the differential equation 

x(t) = Ax(t) + Bu(t) Get) 
and the output equation 

y(t) = Cx(t); (1-2) 
or, as used in this discussion, by a vector matrix difference 
equation, the discrete time solution of equation (1-1) 

x((k+1)T) = QD (T)x(kT) + D(T)u(kT) Gi) 
and the output equation 

y(kT) = Cx(kT). (1-2a) 
Here u is an m dimensional vector (m=n) denoting the control 
imputamcto the plant; y 2S aepedimensi@oenal vector (pe) 
denoting the outputs of the plant; A and P are constant (nxn) 
matrices; B and D are constant (nxm) matrices; C is a constant 
(priematrix. 

The methods required for the evaluation of the transition 
matrix, D, and the matrix D, from the matrices A and B, and 
the development of equations (l-la) and (1-2a) from (1-1) and 
(1-2), are given in Appendix I. 

It 1S assumed that the control is exerted on the plant by 
means of a piecewise constant Signal whose level can change 
only at a regularly spaced "sampling points" or "sampling 


instants". Thus, 


le 
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ae /(3) —~emtk ie weTetteee (k+1)T 
fae Ori, 25... (1-3) 
where T iS a positive constant called the sampling period; 
t = kT is the kth sampling point. 

It is customary to symbolize the sampling operation by 
means of a special element, the sample and hold element (S.H.E.), 
which converts the continuous Signal u(t) into a sampled 
signal u*(t) as given by equation (1-3). The difference equa- 
tion (l-la) relates the state x((k+1)T) at the (k+1)st sampling 
point to the state x(kT) at the kth sampling point and to the 
control signal, u(kT). Fig. 1-1 is a block diagram representa- 
tion of equation (l-la). The blocks, in general, represent 
matrix multiplication as indicated by the double lined arrows. 

Where convenient, the sampling period, T, will be dropped 
from the notation. It will be understood that the conversion 
from a specific sampling point to an instant in time will 
require the sampling period. Equation (l-la) and (l-2a) will 
now be written 

x(k+1) = P(t) x(k) + D(T) uk) (1-1b) 
Vito) Coats) (1-2b) 

It should be noted that the physical quantities repre- 
sented by the state variables and output variables are contin- 
uous functions of time. The values of the functions at the 
sampiang instant are x(k), etc. For convenience and where no 


ambiguity results, the notations x(t), x(k), or simply x may 


13 


be used interchangeably. This will hold also for command 
inputs, o(t) ands (e)m 

To optimally control a plant governed by equation (1-1la) 
it is desirable to express the control input u(k) as a function 
of the state and of all present and future desired states. 
The concept of "desired state" is a generalization of what 
is usually called "command input" or "desired output". If 
all of the state variables cannot be measured directly, and 
if the plant is "observable", it is possible to estimate the 
values of unmeasurable variables through use of the Kalman 
Filter [2 2| . It will be assumed that all of the state 
variables are available. 

In general terms, the design of optimum control involves 
the determination of a control law or a control sequence so 
that the control process can be taken from a given initial 


state, x(0), to a desired state, x., in some finite time. A 


q! 
“control law", wfx(k) K] , vs’the control signal! ©xpresscad™as 
a function of the state variables of the plant and possibly 
tame. A “eontrol Saquence”, u(k) “K=0, ie ., LS emer control 
Signal expressed as a function of discrete sampling instants. 
1.3 The Performance Criterion 

Tf a system is to be optimized, some criterion must be 


chosen that determines how well the system is operating. The 


criterion established depends upon the performance require- 


ments of the system. The method used here is to assign a 


14 


cost to the system at each sampling instant. The sum of 
these costs over the period O=t=NT is called the performance 
index and one form is given by 


N 
Jy on ox(k) + ul (k-1)Ru(k-1)| (1-4) 


where N is the number of sampling intervals to accomplish the 
desired state transition. The matrix Q and the matrix R are 
weighting factors which allow the relative cost of the errors 
and the control effort to be adjusted to achieve the desired 
performance. For optimal design, the performance index is to 
be minimized. The performance index will be discussed in 
detail in Chapter 3. 
1.4 The Problem Statement 

The problem can now be stated: Given the system defined 
i Sequeicrem (1—-a)y find thes control sequence u(0),u(l),%.. 
--,u(N-1) Such that for any initial state x(0) and some 
desired output, a performance index of the form given by 
equation (1-4) is minimized. Furthermore, determine if a 
stationary, linear feedback control law can be established 
such that the total cost over some period is optimum or "close" 


EO Opelmum. 


Ls 


2. TIME OPTIMAL OR DEADBEAT RESPONSE 

2.1 Problem Definition 

It is desired that the system achieve egqulibrium with 
zero steady-state jenronpetrom any init@wal state je @muckly 
as possible. In this and following sections, in the interest 
of simplicity, the desired state of the plant is identically 
zero for all t. It will also be assumed that the plant has a 
Single input (m=1), i.e., u(k) is a scalar. . 

er okder words, the object of control is to transfer any 
initial stagemeai0) sto the equilibrium, state x(N)=0 ssa 
minimum number of sampling periods. It is desired to find the 
control sequence, u(0O), u(l),...,u(N-1), and the control law 
which will accomplish this. A derivation of the control 
sequence is given from which a geometric interpretation can 
be obtained. The result is a stationary control law. 

This problem may also be stated in terms of the performance 
index 

Jy = x (N)Qx(N) (2=1) 

where Q is the identity matrix and N indicates the final 
stage of the process. The problem then is to find the control 
sequence such that N is the smallest integer for which J =0 


and x(t)=0 for all t2=NT. Chapter 3 will show a dynamic 


programming  sedmMiGiom of this problem. 
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baie Control labasssty 


A plant is said to be completely controllable if there 
exists a control signal u(t), defined over a finite interval 
bets txt), which moves the process from initial state x(t,) 
to a desired equilibrium state x(t,) in time interval Ciao: 
In the present case the state x<Cep tee the seate Or 

A necessary and sufficient condition for complete control- 
lability in the discrete-time case may be stated as follows: 


An nth order linear discrete-time process with a scalar 


control is completely controllable if, and only if, the 


vectors 

s, =@ (-T)da(T) 

s, =@ (-27)4(7) 

s, =Q (-nT)4(7) (2-2) 
are linearly independent [1] . How these vectors arise will be 
shown. 


Consider first a second-order system (n=2). Any two 
values of the state variables x1 (k) and x5 (k) define a point 
in the x,;, X5 plane which is called the state space of the 
system. Analogously, any point in the state space is called 
a State. The evolution of the system from an arbitrary present 
Seaee (xX, (k), Xo (k) ) can be represented by a curve in the state 


Space, called the trajectory, along which time is a parameter; 


V7 


~ 





Xo(k) ios cr: crite a ee oe - -7 
ae / 
m(k) L--~ I 
2 i | 
ae | 
| all 
i | 
2 So ! | 
m,(k) | 
S, 
x 
x(k) 


Fig. 2-2 THE CANONICAL AND 
ORTHOGONAL CO-ORDINATE SYSTEMS 
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see Fig. 2-1 where the arrow indicates the positive direction 
of time. Since the solution of a linear homogeneous differ- 
ential equation, i.e., u(t)=0, is uniquely determined for all 
finite values of time by the initial state, there is one, and 
only one, trajectory passing through any given point in the 
Xp» Xo plane. The behavior of the system may be thought of 
as the motion of a point along a trajectory. If the point 
does not move at all, the point is said to be an equilibrium 
state. 


If vj, V5 are the unit vectors along the co-ordinate 


Zz 
axeS in the state space, then the state vector at any time 
kT can be written in the form (see Fig. 2-2) 

x(k) = x, (k)v) + x, (k)v9 (Zee) 
Now, from what initial states x(0) is it possible to reach 
the origin of the state space in one sampling interval? The 
transition equation, again, describing the process is 

x(ktl) =@ (T)x(k) + d(T) u(k) (2-4) 
For k=0, equation (2-4) yields 

x(1) =®D (T)x(0) +(T)u(0) (2-5) 
Thus, if it is assumed that x(1)=0, the initial state from 
which this equilibrium condition can be reached in one 
sampling period is, solving equation (2-5) for x(0) and using 
equation (2-2), 


x(0) = -Q(-T)d(T)u(0) = -u(0)s, (2-6) 


Ls 


(A property of the transition matrix is that G(1) = Qhie?),, 
cream Wen) =Q(-1) ; see Appendix I). Equation (2-6) states 
that if u(Q) is not bounded, all initial states x(Q) lying 


alongethe» veetor sy can» be taken to x«x(T)<0 insonessemplaing 


a 
period. The states» satisfying equation (2-6).constituée a 
straight line in the state space. 

The set of all states from which the origin can be 
reached in at most two steps is that set from which a state 


given by equation (2-6) can be reached in at most one step. 


For k=l, equation (2-4) yields 


x(0) = -u(0)G (-T)a(T) - u(1)@ (-27) 4 (7) 
OE 

x(0) = -u(0)s, - u(l)s, (2=3) 
tf the vectors s, and s, are linearly independent, i.e., 
they do not lie along the same line through the origin of the 
state space, they form a basis in the state space. That as, 
any other vector in the state space can be written as a 


linear combination of sj and s,. 


Tt can be shown |1,5] that the vectors 


® (-T) 4 (7) 
D (-2T) (7) (2-10) 


oi 


‘52 
are linearly independent if, and only if 

TL Frey (2-11) 
where gq is any integer and U/is the imaginary part of any of 


the complex poles of the transfer function of the plant. This 


20 


means that the input must excite all natural frequencies 
of the plant; in other words no cancellation of poles must 
be possible in the transfer function. In illustration 


consider a second-order process with the transfer function 


i 
oe) = Ge 


From a flow graph of this process (see Chapter 3), or from 


the state equations 
Xy =U/X, 
x, _ -U/x, 
and the methods in Appendix I, the following matrices result 


cosG/t sint/T -cosL/T/L) 


@m) = ao = 
-sinU/T cosl/T sin(/T/./J 


Using equation (2-10) 


-1/{/ (-cosL/T cos2U/T - sinl/T sin2l/T)/A/J 
a =2 = 


0 (-cosL/T sin2U/T + sinl/T cos2U/T) 4 / 
By inspection it is seen that if TU/= q7[, where q is an 


Integer, je. = + Ss 


1 and hence the two vectors are not linearly 


2 
independent. 

Tf the values of the control signal, u(0) and u(l), in 
e@meeuon (2-9) ane not bounded, all imatial states x(0) 


lying in the plane containing vectors s. and s, can be taken 


1 2 


to the equilibrium state x(2) = 0 in two sampling periods. If 
the values of the control signal are bounded, it would take a 


longer time to reach the equilibrium state. 


Deli 


The vectors s, and Ss. may bewregarded ae eunt® vector 


a z 


along the axes of a new co-ordinate system, x(k) having the 
co-ordinates m, (k) and m. (k) in this new system, (See Fig. 
2-2). This co-ordinate system will be called the canonical 
co-ordinate system. 
2. JJ DeeermMEiat toner seme Control Sequence and Comesol, maw 
The next step is to determine the actual values of thd 
sequence u(0), u(l), which will take an arbitrary initial 
state x(0) Be eae Crigin of the state space. 
Tt was Stated in the previous section that every mmitial 
state can be written in the form: 

x(0) = m, (0) s) + m.(0)s, (2-12) 
where the m.'s are real numbers. If the process can be taken 
froneehemiinwe1al ysibates x(0) to the equiliiwig@m State in two 
sampling periods, it can be taken from the state x(l) to the 
equilibrium state in one sampling period. The state vector 
x(0) ITS Getvesdancnstenal vector andettermstake vector x(T) 
is a one-dimensional vector in the canonical co-ordinate 
system. Then, x(l) may be expressed as 

x(ly = ing (1)S, (ais) 
Equation (2-4) with k=0 is 

x(1) =Q(T) x(0) + A(T) u(0) (2-14) 
Substituting equation (2-12) for x(0), 


x(1) =(t)[m,(0)s, + m(0) Sp] + d(T) u(0) (2-15) 


Ze 


Simplifying using equation (2-10), the definitions of s) and 


Ss and pmeperties of she *tramsit ron matrix, it folMews that 


So. 
x(1) = my (0) H(T)D(-T) A(T) + m, (0) D (T)P (-27) a(t) + A(T) u(0) 
X(1) = mj, (0)d(T) + m,(0)s, + d(T)u(0) 

or 
x(1) = m,(0)s,; + [m,(0) + u(0)] a(t) (2-16) 


Hence, x(1) will have the form required by equation (2-13) if, 
and only if, 

u(O) = -m, (0) (2-17) 
and 

m,(0) = mj, (1) (2-18) 

If x(1) is to be reduced to zero in one step, then 

Ee (Ty x(l) + acr nat) (2-19) 
EUOSteLeuLIngG equation (2-13) for x(1), 

x(2) =D(T)m, (1)s, + A(T) u(1) (2-20) 
Using the definition s, =D (-T) d(T), 

x(2) = [m,(1) + u(1)] 4(z) (2-21) 
Miewoterc wetnleweendition for x(1) to be reduced to zero in 
one step is 

ae) —m, (1) = -m. (0) (2-22) 
which is the desired control signal at t=T. Hence, at every 
sampling instant, the optimal control signal u(k) is given by 

wie — -m, (kK), (k=0,1 for a second-order 


process) (2-23) 
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where in general for any state, 
x(k) = m, (k) Sy + m,(k)s.,, k=OyeL (2-24) 
The effect of the control will now be determined. Substi- 
tuting equation (2-24) for x(k) into equation (2-4), and using 
the optimal control signal as given by equation (2-23), 


x(k+1) =QD(T) [m1 (kK) s, + mg () $2] - 4 (T)m, (k) 


Simplifying, using equation (2-10), the definitions of Ss, 


and s., 


x(k) 


m,(k)d(T) + mj(k)s, - G(T)m, (k) 


x(k+1) 


m5 (k) Ss, (2-25) 
Comparing this with equation (2-24), rewritten with k=k+l, 

x(k+1) = m, (k+1)s, ~- m., (k+1)s, (2-26) 
we see that artes the application of thesgeptimal control 

m, (k+1) = m. (k) (2-27) 
In the general case for an nth order system, equations (2-24) 
and (2-25) are 

x(k) = m, (k) sy + m.5(k) S95 +...+ ees (2-24a) 
and 

x(k+1) = m5 (Kk) sy + m,({k)s, +...+ my, (kK) S_4 (2-25a) 
ana tnereieme 


m, (k+1) =m (k) (2-27a) 


i¢+1 
A geometric interpretation of equations (2-23 to 2-27a) is 


as follows. The initial state of the plant may lie anywhere 


in the n-dimensional state space. After one sampling period 


24 


and the application of a control signal as given by equation 
(2-23), the state can be expressed as a linear combination of 
only n-1l linearly independent vectors eps ae in other 
words, after one sampling period, the initial state always 
lies in an (n-1)-dimensional space. The initial state is 
taken successively into a space of one less dimension during 
each sampling period. Ultimately, the state must be ina 
space of dimension zero which is an equilibrium point in the 
state space. 

Por the %sceond=order case, the initial statewx(0) lifes 


anywhere in the 20S plane. After the applicationmeot vene 


1 
control, the state x(1) lies on the line described by the 
vector s,- The final control takes the state to the origin. 
2.4 Computing the Control Signal 

According to equation (2-23), the optimal control signal 
is obtained by computing the first component in the canonical 
co-ordinate system, m, (x), of the state of the plant at each 
sampling instant. It will now be shown that the optimal 
control is given by 

u(k) = a!x(k) (2-28) 

Witere a 1S an nxl vector of constants, 1.e., the control law 
is stationary. 


Any component of a vector in one co-ordinate system can 


be expressed as a linear combination of the components of 


the same vector in a different co-ordinate system. Thus, 
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it is always possible to write, considering the second- 
order=process, 

m;(k) = aX} (k) + @5X 5 (k) (2-29) 
wherewehe” a. are réal constants (See Pague=2)ee" 1etre Chen 


necessary to calculate these constants to obtain the control. 


Observe that by definition, 


x(k) = x, (k)v) + x, (k)v, (2-30) 


and 


II 


x(k) my (kK) Sj + m.(k) Ss. (2-31) 
In matrix notation, these expressions may be equated and 
written as 
bers prea = | £1! $2] i 
X5 (k) m> (k) (2-32) 


By definition 


a [Dp i-r)acn) | H(-27) a(9)| 


Pil Pyo2 


oy 22 ; | ie 


The basis vectors sj,S5 are related to the basis vectors 
VirNo bY 


[$1 } $2] = |¥ii¥o] [Pia P12 


P51 P22 (2-34) 
This equation states that 
ST a | 
S, = PyoY, + Pao (see Fig. 2-4) (Z=55) 


Z© 





g,x,(k) x, (k) 


Fig. 2-3 VECTOR COMPCNENTS 
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Fig.2-4 COMPONENTS OF THE CANONICAL 
BASIS VECTORS IN THE ORTHOGONAL 
CO-ORDINATE SYSTEM 


ay 


Substituting equation (2-34) in equation (2-32), 

| Za! Yo] x, (k) = [Za ' Za] [P| m4 (k) 

Xo (k) m. (kK) (2-36) 
Note that the vectors v, and v5 are unit vectors in an 
orthogonal co-ordinate system and hence, have the values 

Vv). Se 1 C 

0 a 
Therefore the matrix 
(1 cm ee 
Oa 
is the identity matrix. Solving equation (2-36), 
m,(k)] = [Pp] ~~ x_ (k) 
1 i 

m. (xk) x, Us) (2-37) 
or in general for an nth-order system, 

m(k) = P7+x(k) (2-38) 
where P-~ denotes the inverse of the matrix P =[81! 89! | Sy. 
Tf the system is completely controllable, the required inverse 
exists. Thus, equations (2-23) and (2-38) together define the 


control law ul x (x) ec teteny 


pee 2 in 


@ 


= a a eal (2-39) 
u(k) = -m,(k) = -a'x(k) (2-40) 
where al is the notation for a vector transpose and is the 
First row of the matrix Pp. The optimum control law is 
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therefore constant feedback of the state variables at sampling 
instants. Fig. 2-5 shows the block diagram for an arbitrary 
second-order process implementing the stationary feedback law 
when the desired state is Q. 

The control sequence can also be found as a function of 
tie state x(0). Frome equationsm(2=20)wande(2-27a), the 
following relations may be derived 

neo). = =m, (k) = —m, 1 (0) (2-41) 


Equation (2-38) with k=0 is 


m(0) = P~+x(0) (2-42) 
Bence, 
u = -P7*x(0) (2-43) 
where 
u = [u(0) 
u(1) 
Hu(n-1) (2-44) 


The optimal control was derived here for a second-order 
system but holds for an nth order system where the initial 
state vector can be expressed as. 

22(0) eerie = U(1)s "Si y- wint) Ss, (2-45) 
If the n vectors 

Ee Seana, i = 1,2,...)n (2-46) 
are linearly independent and if the values u(i), i=0,1,..., 


n-l, are not bounded, the initial state x(0) can be brought to 


ae 


/ 
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Ene CQuitDriunestate x(Nj=0 in at most n samples; that is, 
t ,= nt. The control law was shown to be stationary. For a 
derivation with vector inputs see }4] . 
2.5 Arbitrary Polynomial Inputs |5| 

The foregoing discussion shows the derivation of the time- 
optimal control law when the desired equilibrium point with 
zero error is the origin of the state space. The solution will 
now be extended for a follow-up system. This problem is 
equivalent to making the first component, y, (k), of the plant 
OuUEpUL Vector 

bak) = Cx(k) 2) 

equal to some desired output, © (t) or L(k), in such a way 
that the plant is in equilibrium with zero system error. 

Tt T(t) 1s an arbitrary function of time, the problem 
cannot be solved in general. However, when f(t) hassthe=pemiy- 
nomial form 


2 (t) 


Sy G5 -2-) oe ee 
=O; t=O (2-47) 
(where the g, are arbitrary real numbers) the solution can be 
obtained by modifying the solution of the previous regulator 
problem. 
Note that the components of the plant output vector, y(k), 
are the physical variables of the plant which can be measured 


directly. These components are linear combinations of the 


SE 


plant state variables. For example y, may be the sum of the 
state variables x1 and X5. It will be assumed at this point, 
for simplicity, that 

VS (2-48) 

It is known that the optimal control requires knowledge 

of all of the plant state variables. It then seems feasible 
to derive from (t) a command input state vector, r(t), to 
which it is desired to drive the plant state vector so that 


the plant is in equilibrium with zero system error. Then, 


from equation (2—-47)5) form 


meta = tP-1 
r, (t) = Cec) we Jy + got +..et g,, 
(t) dr (t) 2. ¢t (p-1) tP-¢ 
r = = i Ate = 


STs 
a oe = aie r(t) _ (p=) ee a ec (2-49) 


-] 
aD ) 
Also, note that 
N 
dear) - 0 
at? | (2-50) 
Then 
(bie x, (t) 
G 2-51 
x, ( ) ( ) 


Note that equations (2-49) require knowledge of the derivatives 


of the input. This may be difficult to obtain in some cases. 
If the plant contains at least p "free" integrators (with- 


out feedback), that is, the plant has a type number p, then 
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Pies aitferenrctc ral eGuaefons OL the™ pant in terms of the 


state variables are 


2 
Xo = X54 
“Pp pt (22) 


plus other equations which do not involve the state variables 


x pees Xp- Form the modified state variables 


1 

NA = 

3 = X, -fL 

lee 2 2 

N 

x = x -fr 
p Pp P 

IN fae 

<= XxX. , PHi=n (2253) 
i a 


Substituting equation (2-53) into equation (2-52) shows that 
the modified state variables obey the same differential equa- 


. ee. ; A 
tions. Hence, in equilibrium when x=0, 


xX, = Yy 
Xx, = fo 
ae er 
p p 
x, = OO, pei<n (2-54) 
ai 


which shows that the output will be identical with the input 


after, at most, n sampling periods. 
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Thus, the regulator system, which,contains ,at Jieect > fase 
integrators 1S converted to a follow-up system simply by 


taking the difference between the plant state x(k) and the 


> 


input state r(k) to create the modified state x prior to 


multiplying by the feedback coefficients a In matrix 


lj 
notation, then, 


w(k) = -a" [x(k) - x (k)| (2-55) 


™ 


(the components, ie i of vector FY will be zero) . 


aoe 
Fig. 2-6 shows the block diagram for a simple second- 
order plant with one free integrator and, therefore, the input 
r(t) = constant (p=1). Fig. 2-7 shows the block diagram of 
a follow-up system for the general case where u(k) is given 
by equation (2-55). 
The foregoing solution states that the plant must have 
at least p free integrators, that is, it must be at least of 


type nuibeuep 8@neeraer to follow an #mput t(t) that satisfies 


the differential equation 


pA 
dFr(t) _ 9 (2-50) 


atP 
For example, a plant of type number 2 will follow only “step” 
and "ramp" inputs (p=2) with zero steady-state error. However, 
this restriction on the allowable order of the input to a system 
with a given number of free integrators may be relaxed some- 


what as follows. 
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Fig.2-6 FOLLOW-UP SYSTEM EXAMPLE 


c(t) + -R(t}). -—— ult) u(k) 
Sy >a" Fe SHE. >| Plant 





Fig.2-f VECTOR BLOCK DIAGRAM 
OF FOLLOW-UP SYSTEM 


3 


Consider the example of Fig. 2-6 but where the cGmiand 
Input is a "UuNnLt Bape, 
f(t) =¢ 


or, uSing equations (2-49) 


ae ee 
1 (2-55) 
The desired response then is x1 (co) = t, ane x. (00) =e 


Looking at a signal-flow graph representation of the system, 
Fig. 2-8 (disregarding R), we note, however, that there is a 
feedback of x, upon itself through an integrator and gain of 


-d. This means that x, could not possibly hold the steady- 


2 
state Value melee Otated another way 
X5(cCo) = u(eo) -d:x, (ce) 
0 #0 - a:(1) (2-57) 


We can get around this difficulty by introducing an auxiliary 
input, R, as shown in Fig. 2-8. Equation (2-57) could then be 


written 


il 
ye) 
| 
Os 
x 


&, (08) 


0 


{| 
aD 
Qu 

= 


(2-58) 
Therefore, 

R = d-(1) (2-59) 
It is seen, then, that through the use of an auxiliary input 
a plant can be made to follow a polynomial input of one order 


higher than the type number of the plant. The introduction of 
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Fig.2-8 FOLLOW-UP SYSTEM EXAMPLE 
WITH AUXILIARY INPUT 





Fig.2-9 FOLLOW-UP SYSTEM EXAMPLE 
FOR ARBITRARY (IN A CLASS) INPUT 


a7 


the auxiliary input Simply results in a SHYTEC in the or seem 
of the canonical co-ordinate system. The equilibrium state 
is still reached in k<n stages. 

It is seen £rom equation (2-59) that, if the auxiliangy 
input is to be implemented as shown in Fig. 2-8, the command 
input and its derivatives must be known beforehand and may not 
be arbitrary. However, by implementing R as a function of the 
input state vector r, the system will follow an arbitrary, but 
still restricted, polynomial input. 

Continuing with the example of Fig. 2-8, it is seen 
that 

u=- a4 1 (sy - ry) - A, (Xo ~ ro) + R (2-60) 
where 


R 


d-¥r5 (2-61) 
Then substituting for R, and rearranging, 

UR eee + (aj5 + dq) Yo - 841%] - 812%5 (2-62) 
If the system is redefined as shown in Fig. 2-9, and if 


be == and=b, = (a 


1 il + d), it is seen that u is still given 


Ihe 
by equation (2-62). The input vector, r, may now represent 
an arbitrary polynomial, ina class, and the origin of the 

canonical co-ordinate system will be automatically shifted 

so that the equilibrium state may be reached. 


In general matrix notation the control, u(k), is given 


by 


u(k) = b'r(k) - atx(k) (pees 
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where 


Boos lb a as 


b ee Eat = (2-64) 


(the elements b 


p+ “bi, where p<n, are zero and must be 


maeluded in the vectors b and r so that they will be confomni- 
gole with the (nxl) vectors a and x.) Elaboration is required 
mom the general formulation of the vector b. 

Tt can be shown [6] that for a scalar control, u(t), and 
a completely controllable system, the coefficient matrix A 
of equation (1-1) can be written in a canonical form. This 
results in the signal-flow graph representation of any plant 
as shown generally in Fig. 2-10 (output not shown). The 
plant shown has m free integrators. From the preceding example 


it can be seen that the elements of the vector b are formed as 


Follows: 
bs = ay, + Oo: 1 = 1L,..F.m 
DS = 81p + O ; pxmitl 
a = ap + Oa >; p= mtl 
be =-0 > j = ptl,..33n; p=<—n 


where p is the order of the polynomial input and where dtd 
1s shown in Fig. 2-10. The a,, are the optimal gains given 
by the first row of .. 
Tt has been shown that a plant can be made to follow a 
polynomial input function of order one higher than the number 


Gemeree integrators in the plant. This will be illustrated in 


Chapter 4. 


5° 
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Tier seiee Ore eicapmane, x(t), 1S Avconeimucus LUuneETONn 
of time and is being fed back continuously, while the control 
input is constant over the sampling period. Thus, if the 
desired value of any of the state variables is not a constant, 
and if it is being fed back to the input, then the error at 
the input will not be zero at all times and therefore i 
system will not be in eguilibrium. This will also be illustrated 
in Chapter 4. 
If the plant output is other than that given by equation 
(2-48), VY, = X,, the command input vector r must be written 
in a form to correspond with the desired values of the state 
variables. For example if the output of the second-order plant 
shown in Fig. 2-9 is given by 
ee 2" 
and the desired output is a ramp, 
o(t) = gt 


then for zero error 


eos XX (2-66) 
Taking the derivative, 

g = xX, + X, 
or 

Saas) + O; X) = Xx, (2-67) 


for equilibrium. Solving equations (2-66) and (2-67) simultan- 


eously, 
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= t—] = 
xy g ( ) ry 


X, = 9 = Ly (2-68) 
Equation (2-68) gives the desired values of the state variables 


and, hence, the command input vector. 
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3. GENERALIZED PERFORMANCE INDICES AND DYNAMIC PROGRAMMING 
3.1 General 

In this chapter dynamic programming will be applied to 
optimize the performance of a dynamic system. The results 
are applicable to both sampling and continuous systems, a 
fact which will be discussed later in the chapter. This 
approach leads to a stationary control law for the infinite 
stage regulator problem. 
omc Problem D@@inition and Development 

Consider the system discussed in Chapter 1 described 
by the difference equation 


x(k+1) =D, (T) x(k) + DL (T)u(k) (3-1) 


A performance criterion is established to measure "how 
well" the system's output follows the command input T(t). 
This is called the performance index and is denoted by J. 
Fig. 3-1 shows schematically how the performance index is 
evaluated. 

Inputs: The method of dynamic programming requires the 
knowledge of all the desired states; therefore, the system 
will be optimized for an expected class of command inputs 

(2 ce) A member £(t) Of thisvclacecesot inputs sis defined by 


T(t) 


r, (t) > t=O 
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where 


x(t) =@ (t)w(0) = [x (t) 


SS (3-3) 
PD, (t) 1s the transition matrix of a pth order linear differential 
equation with constant coefficients and w(0) is an arbitrary 
vector. PD, determines the class of inputs for which the system 
is to be optimized and w(0) determines a particular member of 
that class. 
Example 1: To illustrate the formulation of @. Suppose a 
member of the class of inputs (2(t)} satisfies the differential 


equation 


2A 
d rit) = 0 (3-4) 


at? 


The equations for the state variables, (2-49), lead to the 
flowgraph shown in Fig. 3-2. PD, may be determined by inspec- 
tion of the flowgraph: 
-1 2 a 
Wee =e ifs 17s) et 
0 1/s Ol (3-5) 
Then, from equation (3-3), 
a = 0) + tw_(0 3-6 
r, pes wt CO) A ) (356) 
Example 2: The solution to the harmonic equation 
ee ae 0... t =O (3-7) 
amet 2 = 
yields a sinusoidal signal. This equation may be decomposed 


into 
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Fig. 3-2 FLOWGRAPH FOR POLYNOMIAL 
EXAMPLE 





> w (0) 


Fig. 3-3 FLOWGRAPH FOR SINUSOIDAL 
EXAMPLE 
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vapmee a 
mS = -LJ/wy (3=8) 
From the state equations or from the flowgraph shown in Fig. 


3-3, QD. is seen to be 


D ,(t) a ae we 


| s7 +tJ* 5? ie 


eos t emai) tt 
-sinlJt cosl/t (3-9) 


Then 


r, (t) w, (0) cost/t + w. (0) sint/t (3-10) 


If a system has been designed for optimal performance for 
a particular class of inputs, it will approach nearly optimal 
performance with any other input signal which can be closely 
approximated by an r(t) over one sampling interval {3} - 
Performance Index: The error between command input and plant 
output is given by 

e(t) = F(t) - y, (t) (3-11) 

The undesirability of error is not usually a function of the 
sign, so a more plausible measure would be 


e(t) 


jE (t) - y, (e) 


Or 


e(t) = (F(t) - y,(t))* (3-12) 
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The second is commonly used because of its mathematical 
convenience. A generalized measure of error is defined as the 
quadratic form 

Ak 

Zz (t)Ont eyo (3-153) 
where Q is an (n+p) by (n+p) positive semi-definite matrix, 
and Z(t) 1s an (ni) See vector deétrmmed by 

ZAEy Sea) 

x(t) ~ (3-14) 
Example: To™mmllustrate the preceding development, stippose 
the desired measure of error is 
‘: Z 
(xr, ) - x, (t)) 


Then, the Q matrix is given by 


ieee; — 2 


Taint) ~ Sn+1,1 ~ 7 
all other elements = 0O (3-15) 


To complete the notation in terms of the combined state vector 


ace, ;, erie 


D(T) Dayal T) 


D.(T) (3-16) 


P(t) 


i 
© 
* 

a 
© 


0 ! (T) (3217) 
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where D (TY, DL (7) and (1) are as previously defined, and 
D (T) Poe. Saniee aera rol Mows™irom the derinition of F(t) 
in equation (3-3) where there is no DT) term. 
The two state transition difference equations are com- 
bined as 
z(k+l) =Q(T)z(k) + D(T)u(k) (3-18) 
In many systems the control energy represents cost and, 
therefore, it may be desired to consider control energy in the 
measure of performance as well as the system's errors. The 
quadratic form is also used for measuring control effort 
uw’ (t)Ru(t) 
As stated previously, if the performance of a system is 
to be optimized, a criterion must be selected to tell "how 
well" it is operating. This criterion is the performance 
index, which, over the period 0S t= NT is given by 
Ty [Z(0) -U(T), «+ 2 (N-1) 7) 


NT 

=| [ z(t) Qz(t) + u*' (t)Ru*(t)) (t) dt | (320) 

0 
where Q and R are proportional weighting matrices which com- 
promise between minimization of errors and control energy. 
A(t) is a positive time-variant weighting function. Note that 
the smaller Ty the more satisfactory the system's performance. 

The foregoing definition is very general and by varying 

Q,R and X(t) the index is adjusted to fit problems of physical 


significance. 
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Example l: The response of a plant is never instantaneous, 
and as a result for some period of time following the applica- 
tion of the control signal, errors are inevitable. As time 
progresses, these errors are expected to become smaller. 
Therefore, it may be desirable to have A(t) increase with 
time. Ifthe errers and the control Gmtert are to be measured 
only at the sampling instants and given zero weight at all 


other times, then, “i 


or = 


N Ax) (z" (k)Qz(k) + u* (k-1)Ru(k-1) J (3-20) 


1 Mz 


il 
where X(k) is defined as the value of A(t) at the kth sampl- 
ing instant. If the weighting of errors and control effort 
remains the same at all sampling instants, then ) (k) is 
dropped from the performance index. 

Hxample 2: Awoer Formance index Conecemnecd emia cas Ciaidai 


at the Nth sampling instant is 

J = 2 (N)Qz(N) (3-21) 
If Q is the identity matrix, this index is the Euclidean 
norm of the system's state variables at the end of the Nth, 
or final stage of the process. Note that it reflects no 
concern whatever for what goes on in the early stages of the 
process, nor for the control effort required. It reflects 


the desire to bring the system as close as possible to the 


Ov renin... 


ae 


Tf a system has deadbeat response as described in Chapter 
2, it is possible to select a control signal that will make 
Jy, = 0 for a finite N and any initial state z(0). 
3.3 Recurrence Relations |7| 

It is now possible to state the problem more precisely. 
Given a linear plant defined by equation (3-1) and a class of 
input signals defined by equations (3-2) and (3-3), find the 
Somebea control signal u(k) for the t=O which minimizes the 
performance index. That is, find the sequence of vectors 
en, Uti ul2),... such that for any initial state, zi0), 


the performance index J equation (3-19), is minimized. The 


Nic 


performance index considered is 


N T mT 

5{z(0)) =), [2" (x) 92 (4) Pan (k-1)Ru(k-1) (3-22) 
k=1 

which is the case where Ak) is dropped from equation (3-20). 


The optimum return function Ty 2(0)! is defined as@tne 


minimum value of Ty [2.(0)] and has the known simple form 
5°(z(0)) = z (0)P.2(0) (3-23) 
ul2 y= IS 


which is proved by induction later. Pay is a symmetric, 


positive semi-definite matrix. By definition 
min min 
Tyr [2(0)] = u(0),u(1),.. 
N+1 


min 
_ a(n) Xe [27 09 02.00) + u" (k-1)Ru(k-1)| (3-24) 


all 


Z2(1) is determined solely by u(0), which enables equation 


(3-24) to be written as 


min min 
Tye [2 (0) | > (OV) 2 (1)Qz(1) + u*(0)Ru(O) + u(1), 
min min N+1 = o 
152) eee Na). | Z (Te QZ act aa c1)R(-2)]} 
ie— 2 
(3-25) 


The last group of terms is exactly JR (2()] which gives 


mer) 


O 7 T 7 7 O 
Ty41 [2 (| =e) e (1)Qz(1) + uv (O)Ru(O) + sefzcy} 
(3-26) 
This equation can be arrived at directly using the 
principle of optimality, which states that the minimum cost 
of an N+l stage process is the minimum of the sum of the cost 
of the first stage and the minimum cost of the N remaining 
stages. Note that the arguments of the state and control 
variables increase with time, whereas the subscript on Jn 
decreases with time. 
Substituting equations (3-18) and (3-23) into (3-26) 
Tyr [2 (0) | = 
min m 
= wConleco) + Du(0}} 7(Q + Pp ) [pz00) + Du (0)| + u (0)Ru(0) 
= = = N 
(3-27) 


Completing the square on the right side of (3-27) and 
defining 
WOO eee) (3-28) 
= -|[D"(Q + P_)D + R]~"D" (Q + Pp 1D (3-29) 
Au+1 N N 


a2 


Equation (3-27) becomes 


min 


2(oH[2(0 - u' (0}]  ( ay R][ (0) oe (0)| 


F41[2()] 


+ 


2 (0)D- (0 “ P P20) 


2 (0)A,,,[D™(0 + PD + R| Aa 200) (3-30) 


The control u(Q) occurs only in the first term. If the matrix 
[D™ (Q + P.)D + R| is positive definite the optimal control is 
unique. Then, the minimum value of the first term is zero 
and occurs when 
co) = u’ (0) (3-31) 

The matrix will be positive definite if R is positive definite, 
or if Q is positive definite and the columns of D are linearly 
independent. [It will not be positive definite if R = QO, and 
the columns of D are linearly dependent. In other cases the 
matrix might be singular. 

Equations (3-31), (3-28) and (3-29) define the optimal value 
of a (0) .. 

The recurrence relation for P el is determined by equating 


(3-30) and (3-23) when u(0O) = u' (0), 


Tyoq [ZC] = ZOD (2 + Py P20) 


Z™(O)A w41[D’ (0 + Py) D + R] Ayy12(0) 


Ze CONE, ZO (3-32) 
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Since equation (3-32) must hold for all 2(0), the recurrence 


relation becomes 


Pry =P (Q + Py) Dt DAyyy) (3-33) 
where 

a AT 41 ([D (+ PD + R| =O 7 Px)D (3-34) 
on 

a -[p* (Q + Py)D + R| “1p? (Q + PID (3-34a) 


has been used to simplify (3-33). 


O 


Equation (3-32) shows that if the quadratic form for Jn 


is correct, then oer has the same form. The quadratic form 


fomerrVvially=eerrect for I5 since 
O = 
35 [z(0)| = 0 (3-35) 
for all z(0)- “To complete the mathematical induction, the form 
Po a must be shown Co be correct. aT is determined from 
equation (3-32) and (3-29) noting that = O. Equation (3-34) 


1S again used to simplify the result. 


Jf [2] z7 (0) 'a (P - v(vTap + R) ~~ Dog]z(0) 


z’ (0) P4z(0) (3-36) 
Ty [ 2 (0) hadswenemiestred quadratic Form. 
3.4 Solution of the Problem 

The solution proceeds using the recurrence relations, 
equations (3-33) and (3-34a). Since J5{2(0)] = (0, Py = o> 


calculate Aj- From Ay and Po calculate P,- Continue this 
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process until all a of interest are calculated. If the 

plant is controllable, An will tend to a limit as N increases. 

Therefore, for the infinite stage regulator problem the 

Optimal control takes the form of a stationary linear function 

of the states, which is independent of the sampling instants kT: 
u(k) = Agz(k) (3-37) 


However, if it is desired to optimize J, over a finite number 


N 
of samples, then the optimal control takes the form of a 
different linear function of the states at each sampling 
instant. 

u(k) = A(k)z(k) (3-38) 

An advantage of this design procedure is that it always 
results in a stable control system. This is due to the state 
method of describing the system's dynamics, and to the use of 
dynamic programming to obtain the solution Bye 

The resulting system is shown in Fig. 3-4. Note that 
it is a multiloop feedback system, requiring measurement of 
the plant's states as well as the input states. 

If T becomes small, relative to the plant's time constants, 
the sampled signal u*(t) is nearly the same as the continuous 
slonakeu(t). Then the sample and hotd element can be removed 
from the system, resulting in a continuous system. If R= 0, 
and if T approaches zero, then the control input becomes 
unbounded. For this reason R must be greater than zero if T 


is going to be very small, or if a continuous system is desired. 
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A comment on notation is made here. When the control 
iapuc u is @ vector quantity the feedback control Matrix will 
be denoted as Ay whereas, 1£ the control input is a scalar 


Guantity, whe feedback control matrix will be denoted as 


at 
—N 


a7, 


4. NUMERICAL EXAMPLES 
To illustrate the gontrol laws derived ang@bapters 2 
and 3, the following examples are presented. All simula- 
tions were done with the CDC 1604 digital computer. 
4.1 Deadbeat Response 
The third order system shown in Fig. 4-1 was simulated. 
The descriptive equations are 
x(k+1) =Qx(k) + du(k) . 
ute, = Be cee) 
Ne 
The state transition matrix®, and the d matrix for a sampl—- 
ing period T = T/4 are 
Q=f1 .7091 .2509 
Pee 1205 ee DGD 
0 -—.6088 .3914 
dad = ; .06965 
509 
Looe 
Using the deadbeat performance index 
aia x'(N) I x(N) 
and the recurrence relations derived in Chapter 3, the 
following control law was computed, where, due to the use of 


a scalar control, N is equal to the order of the plant. 


u(k) ax (k) = fay ao = x(k) 


|-2.741672 -3.127373 -2.045130] x(k) 
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The same gain matrix was obtained using the methods of 


Chapter 2. The system's trajectory is presented in Table 4-l. 


faa @ | eae [ome [6 
[a [eas |_ aoe [ann | ae 
ae Te 


TABLE 4-1 SYSTEM TRAJECTORY 
FOR DEADBEAT EXAMPLE 













The recurrence relations also yield a series of gain 
matrices forming a control law which is a different function 
of the states at each sampling instant, 

u(k) = a° (k)x(k) 
This control law gives the identical control sequence u(0O), 
u(1),...,u(N-l1) as the stationary control law above. 
4.2 )Infinites Seage Regulator 

The optimum control law for the infinite stage regulator 
is also in the form of a stationary linear function of the 
state variables. The control gains and the cost function can 
be computed byeenmetrecurrence relations of*Chapter Gem AfEter 
the optimum gains are established, the system's trajectory is 
computed from the difference equation of the system. 

This example utilizes the same third order plant as the 


deadbeat example, Fig. 4-1. The problem was formulated as 


60 


in Chapter 3 and the resulting system is as shown in Fig. 3-4. 


The performance index is 


N 
T 
Jy =) [2 (k)0z(K) + = (k-1)R| 
a 
where 
o=| 10 0-1 O R = 1.0 
0 1 Owned p= 1/4 


Or=1-. CG: 

For a polynomial input such as described in equations (3-4) 
through (3-6), the gain matrix was computed to be 

ay = | ~.581355 ~.626587 -.922714 .581355 1.716587] 

The command inputs used were a unit step and a unit ramp, 

and the results are shown in Fig. 4-2 and 4-3 respectively. 
Both step and ramp responses were within 1% of the desired 
values at the end of eleven samples. Note that the gain 
Matrix has the pattern expected for a system with one free 
integrator as discussed in Chapter 2; i.e., that a = —-a 


n+l il 


and a = -(a 


ha + d) = -(a, - 1.09). In the development of 


2 
Chapter 2, it was shown that such a system would reach equilib- 
rium with step and ramp command inputs only if in addition to 


the optimum plant state feedback gains, optimum gains on the 


command input states were incorporated; i1.e., if the optimum 


gains are not applied to the input states, the system's ramp 
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response will have a steady state error. A ramp command 
input was applied to the example's system, without the gains 
on the input states, and the response is shown in Fig. 4-4. 

For a Sinusoidal command input as developed in equations 
(3-7) through (3-10), the gain matrix was computed to be 

an = | -.581355 -.626587 -.922714 .37783 581434 | 

Note that the first n elements are the same as for the poly- 
nomial command input, but aol and avo are different. The 
system's trajectory is shown in Fig. 4-5. 

Fig. 4-6 shows the trajectory for the same system with 


a Sinusoidal input but with the weighting matrix Q aS given 


by equation (3-15). The gain matrix was computed to be 


an = [-4.484 -4.275 -2.313 4.080 1.815] 


4.3 The Finite Regulator Problem (CASE STUDY) 

As stated in Chapter 3, if it is desired to optimize 
a system's performance over a finite number of samples M, 
where M is either smaller or not much greater than the number 
of recurrence steps required for Ay to tend to a limit, @he 
resulting control takes the form of a different linear func- 


tion of the states at each sampling instant, 
u(k) = A(k)x(k) 
A problem of this type would occur if the command input to the 


system were reinitialized every M samples. An example might 


be a missile that is using a search-while-track radar and is 
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receiving new information at the rate MT, where T is the 
system's sampling period. 

To physically implement such a system would require a 
computer to reset the control feedback gains at each sampling 
instant. The engineering complexity involved with such a 
system warrants investigating a system with fixed control 
feedback gains. Though not optimal, this system might be 
considered superior due to other criteria such as weight, 
cost, etc. This form of control will be referred to as sub- 
optimal in this section. 

An investigation of suboptimal control was accomplished 
by simulating second, third and fourth order systems and 
comparing the performance to optimal. The performance index 
used was 


WJ aa 


N (3c (ic) ax (K) + a? (k=1)R] 


ae 


i 
and most of the investigation was made with the second order 
system of Fig. 4-7. 

The state transition matrix ®, and thesdewatrix for & 


sampling period of 0.1 are 


P 
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The weighting matrices are 


o= fi 
ol 
R = [1.0] 


The system was cum ner Live samples. 

Tt was observed that if the system was run for five samples 
using A. as the fixed control feedback matrix, the performance 
was not greatly degraded from that of optimal. This was also 
true of the higher order systems. The amount of performance 
degradation that is acceptable is, of course, an engineering 
decision based on the physical aspects of the problem. 

Runs were also made using the values that Ay tended to 
in the limit. Even though these values gave optimal perform- 
ance in the infinite stage regulator problem, the total cost 
was higher in the five-stage finite regulator problem than 
that obtained using the gain matrix A. 

The elements of A, were then perturbed to see if the 
performance could be improved. This led to an exhaustive 
search which resulted in a gain matrix which differed signifi- 
Gantly Fran Ac, and the resulting performance was near optimal. 
During the search the system's parameters and the initial 
conditions were held constant. 

Using the gains obtained in the search, various initial 


conditions were tried, generally resulting in performance 
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close to that obtained with A... The results of this case 


study are presented in Table 4-2. The column A_ contains 


the costs using gains obtained by perturbation. 


Initial Metal Cogt Jiee> [x () Tx (k) + u* (k-1)] 
Conditions 7 


af ila 
A Aco A 
5 p 
3 
48.658 | 48.712 | 55.150 | 48.7714 


aeu210 8.0500 Jo 5956 8.0214 
28.0741 2852912 34.2085 28.1898 


Z50%eell 253% lest 270.0404 233a7e7 





eer 
A 
a 
3 
10 
488.0655 488.2655 5 oie LS 490.7636 | 
2 
10 


TABLE 4-2 RESULTS OF CASE STUDY OF SUBOPTIMAL 
CONTROL FOR THE FINITE STAGE REGULATOR PROBLEM 


* Initial conditions used during exhaustive search 


Gain Matrices: 


— [-.038279 -.264931| 
a |-. 376449 -.509776] 
A = .045198 -. 302418] 


@peamal (Gn oder of application): 


|-.038279 -.264931| 


A 
5 


Ay 


|--021133 = 243e0e) 


hid 


|-.007999 -.216953| 
.000587 -.183544| 


004439 -.139468| 
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5. CONCLUSIONS 
9.1 Summary 
»>.-l.1 Deadbeat Response 

The optimum control law derived for deadbeat response 
takes the form of a stationary linear function of the state 
variables, independent of the sampling instant kT: 

u(k) = A.x(k) 
where N is the number of stages required to bring the system 
to the origin, N=n. 

It has also been shown that by using dynamic program- 
ming to minimize the performance index 

J = x*(N)Qx(N) 
whe rewa(@ijedsethe final stage vector and Q is the identity 
matrix, the resulting control sequence will give deadbeat 
performance. The control law may be either stationary or a 
different linear function of the states at each sampling 
instant kT. The control sequence is the same in both cases. 

u(k) = A.x(k) = A(k) x(k) 

Decent ommeeenate Stage Regulator 

The optimal control law derived for the infinite 
stage regulator is in the form of a stationary linear func- 
tion of the state variables. The procedure for obtaining the 
constant control gains is well established ee 7 and 1S given 


in Chapter 3. 


ie 


5.1.3 The Finite StagesRequibator 


The optimum control law derived for the finite 
stage regulator is a different linear function of the state 
variables at each sampling instant, 

u(k) = A(k)x(k) 

If it is desired to achieve the best performance 
possible with a stationary control law, the results of a 
case study tend to indicate that using Ay as the fixed gain 
matrix, where M is the number of sample periods, results in 


a system's performance not greatly degraded from optimum. 
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APPENDIX I 


Derivation of Difference EquationsmRepresent ing 
a Sampled-Data Dynamic System 


A block diagram of the system discussed in this Appendix 
is SHOWN” in Firioe” las 
Plant: A linear, time invariant dynamic system with state 
equations, which in vector notation may be written 


X(t) = Ax(t) + Bu(t) i (qi 1.) 


vace) Cxue) Pept ) (I-2) 
The Laplace transformation of equation (I-1) is 


sx(s) -= x(0-) 


AX(s) + BU(s) (I-3) 


Rearranging, 


StS) FR a aealO-) +. BUS) 


(sI - A)X(s) = x(0O-) + BU(s) 


X(s)l= (ste-ea)x(O-) + (sz - A) BU(s) (1-4) 


Let 


@(s) = (st -a)~* 
Pit) =L“[Prs) 


The solution of (I-1l1) becomes 


ie 
x(t) =G(t)x(0-) + [Pe Batra (1-5) 
0 


where it is assumed that u(t) O; t<QO, and P(t) is causal. 


More generally: (initially t ty vice Zero) 


LO 


= 
malic) = P(t-t.) x(t) [ben Bu (T) aT (I-6) 
t 


Se 


natural response forced response 


For an interval of time T set: 


to = kT 
t = (k+1)T 
Then, (I-6) becomes: 
(k+1)T 
(ile 1,).7) =()(T) x(kT) +f D ( (x41) -7) Bu(T) dT 


ea (1-7) 


where u(t) is. pi@cewise constant, il.e., 
u, (t) = u, (kT) 


u, (t) = u, (kT) 
for kT=t <(k+l)T 


u(t) u,, (KT) 


Then, 


(k+1)T T 
{ h ((k+1) 7-1) Bu (T) aT = [een Ba7T$u (kT) 
kT 5 


= D(T)u(kT) 
Then, (I-7) becomes 
x(k+1)T = (T)x(kT) + D(T)u(kT) 
Some Properties of the State Transition Matrix 


1) (0) is an identity matrix 


1 


2) peer) = Grr) ® 
Since by definition, P(kT) =e 


3) Q(-ktT) = (pi«r) )~ 


Since@ (-kT) = ego pape ow 


ATk k 
=. en) 
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